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Abstract 



We present an extension of sparse PCA, or sparse dictionary learning, where the sparsity 
^ , ■ patterns of all dictionary elements are structured and constrained to belong to a prespecified 

set of shapes. This structured sparse PCA is based on a structured regularization recently 
introduced by [1]. While classical sparse priors only deal with cardinality, the regularization 
^ \ we use encodes higher-order information about the data. We propose an efficient and simple 

optimization procedure to solve this problem. Experiments with two practical tasks, face 
recognition and the study of the dynamics of a protein complex, demonstrate the benefits of 
the proposed structured approach over unstructured approaches. 
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^ ■ 1 Introduction 

p ■ 

Principal component analysis (PCA) is an essential tool for data analysis and unsupervised di- 
mensionality reduction, whose goal is to find, among linear combinations of the data variables, a 
sequence of orthogonal factors that most efficiently explain the variance of the observations. 
5^ \ One of its main shortcomings is that, even if PCA finds a small number of important factors, 

the factor themselves typically involve all original variables. In the last decade, several alternatives 
to PCA which find sparse and potentially inteipretable factors have been proposed, notably non- 
negative matrix factorization (NMF) [2] and sparse PCA (SPCA) [3, 4, 5]. 

However, in many applications, only constraining the size of the factors does not seem appro- 
priate because the considered factors are not only expected to be sparse but also to have a certain 
structure. In fact, the populaiity of NMF for face image analysis owes essentially to the fact that the 
method happens to retrieve sets of variables that are localized on the face and capture some features 
or parts of the face which seem intuitively meaningful given our a priori. We might therefore gain 
in the quality of the factors induced by enforcing directly this a priori in the matrix factorization 
constraints. More generally, it is desirable to encode higher-order information about the supports 
that reflects the structure of the data. For example, in computer vision, features associated to the 
pixels of an image are naturally organized on a grid and the supports of factors explaining the 
variabihty of images could be expected to be localized, connected or have some other regularity 
with respect to the grid. Similarly, in genomics, factors explaining the gene expression patterns 
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observed on a microarray could be expected to involve groups of genes corresponding to biological 
pathways or set of genes that are neighbors in a protein-protein interaction network. 

Recent research on structured sparsity [6, 7, 1] has highlighted the benefit of exploiting such 
structure for variable selection and prediction in the context of regression and classification. In par- 
ticular, [1] shows that, given any intersection-closed family of patterns V of variables, such as all 
the rectangles on a 2-dimensional grid of variables, it is possible to build an ad hoc regularization 
norm Q that enforces that the support of the solution of the least-squares regression regularized by 
belongs to the family V. 

Capitalizing on these results, we aim in this paper to go beyond sparse PCA and propose 
structured sparse PCA (SSPCA), which explains the variance of the data by factors that are not 
only sparse but also respect some a priori str uctural constraints deemed relevant to model the data 
at hand. We show how slight variants of the regularization term of [1] can be used successfully to 
yield a structured and sparse formulation of principal component analysis for which we propose a 
simple and efficient optimization scheme. 

The rest of the paper is organized as follows: Section 2 introduces the SSPCA problem in the 
dictionary learning framework, summarizes the regularization considered in [1] and its essential 
properties, and presents some simple variants which are more effective in the context of PCA. Sec- 
tion 3 is dedicated to our optimization scheme for solving SSPCA. Our experiments in Section 4 
illustrate the benefits of our approach through applications to face recognition and the study of the 
dynamics of protein complexes. 

Notation: For any vector y in W and any q > 0, we denote by \\y\\^ = (l^j=i \yj\°^Y^°^ the 
(quasi-)norm la of y. Similaiiy, for any rectangular matrix Y G M"^^, we denote by ||y||p = 
(Sr=i Si=i ^ij)^^"^ its Frobenius norm, where Yij is the (i, i)-th element of Y. We write Y^ for 
the j-th column of Y. Given w in W and a subset J of {1, ... wj denotes the vector in W 
that has the same entries Wj as w for j G J, and null entries outside of J. In addition, supp(w) = 
{j G {1, . . . ,p} ; tLij / 0} is referred to as the support, or nonzero pattern of the vector w G M^. 
For any finite set A with cardinality \A\, we also define the |A|-tuple {y'^)a&A £ M^^l"^! as the 
collection of p-dimensional vectors y° indexed by the elements of A. Furthermore, for two vectors 
X and y in R^', we denote hy x o y = {xiyi, . . . , Xpyp)~^ G the elementwise product of x and 
y. Finally, we extend | by continuity in zero with ^ = oo if a 7^ and otherwise. 

2 Problem statement 

It is useful to distinguish two conceptually different interpretations of PCA. In terms of analysis, 
PCA sequentially projects the data on subspaces that explain the largest fraction of the variance of 
the data. In terms of synthesis, PCA finds a basis, or orthogonal dictionary, such that all signals ob- 
served admit decompositions with low reconstruction enor. These two interpretations recover the 
same basis of principal components for PCA but lead to different formulations for sparse PCA. The 
analysis interpretation leads to sequential formulations ([8, 9, 3]) that consider components one at 
a time and perform a deflation of the covariance matrix at each step (see [10]). The synthesis inter- 
pretation leads to non-convex global formulations ([4, 11, 9, 12]) which estimate simultaneously 
all principal components, often drop the orthogonality constraints, and are referred to as matrix 
factorization problems ([13]) in machine learning, and dictionary learning in signal processing. 

The approach we propose fits more naturally in the framework of dictionnaiy learning, whose 
terminology we now introduce. 
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2.1 Matrix factorization and dictionary learning 



Given a matrix X G M"^^ of n rows corresponding to n observations in W, the dictionary learning 
problem is to find a matrix V G R^^^', called the dictionary, such that each observation can be well 
approximated by a linear combination of the r columns {V^)ke{i,...,T} of ^ called the dictionary 
elements. If f/ G M"^'' is the matrix of the linear combination coefficients or decomposition 
coefficients, the matrix product UV~^ is called a decomposition of X. 

Learning simultaneously the dictionary V and the decomposition U con^esponds to a matrix 
factorization problem (see [5] and reference therein). As formulated in [14] or [5], it is natural, 
when learning a decomposition, to penalize or constrain some norms or quasi-norms of U and V, 
say Qu and Qy respectively, to encode prior information — typically sparsity — about the decompo- 
sition of X. This can be written generally as 



1 

mm 



x-uv~^: 



+ X'^n„{V'') s.t. yk, QuiU'') < 1, (1) 

k=l 



where the regularization parameter A > controls which extent the dictionary is regularized ^ If 
we assume that both regularizations Uu and Qi, are convex, problem (1) is convex w.r.t. U for V 
fixed and vice versa. It is however not jointly convex in {U, V). 

The formulation of sparse PC A considered in [12] corresponds to a particular instance of this 
problem, where the dictionary elements are required to be sparse (without the orthogonality con- 
straint V^V = I). This can be achieved by penalizing the columns of by a spai^sity-inducing 
norm, e.g., the £i norm, Qy{V'') = ||^^^||^- In the next section we consider a regularization O^, 
which controls not only the sparsity but also the structure of the supports of dictionary elements. 



2.2 Structured sparsity-inducing norms 

The work of [1] considered a norm which induces structured sparsity in the following sense: the 
solutions to a learning problem regularized by this norm have a sparse support which moreover 
belongs to a certain set of groups of variables. Interesting sets of possible supports include set of 
variables forming rectangles when aiTanged on a grid and more generally convex subsets^ . 

The framework of [1] can be summarized as follows: if we denote by ^ a subset of the power 
set of {1, . . . ,p}, such that UggS G = {1, . . . ,p}, we define a norm on a vector y G M^' as 

Geg ^ jeG ^ Geg 

where {d'^)G<^g G M^^'^' is a |^|-tuple of p-dimensional vectors such that > if j G G and 
= otherwise. This norm Q. linearly combines the I2 norms of possibly overlapping groups of 

variables, with variables in each group being weighted by {(P)Geg- Note that a same variable yj 

belonging to two different groups Gi , G2 G ^ is allowed to be weighted differently in Gi and G2 

(by respectively dj^ and d°'^). 

For specific choices of Q, Q. leads to standard sparsity-inducing norms. For example, when Q 

is the set of all singletons, 17 is the usual li norm (assuming that all the weights are equal to 1). 

'From [14], we know that our formulation is also equivalent to two unconstrained problems, with the penalizations 

■^We use the term convex informally here. It can however be made precise with the notion of convex subgraphs 
([15]). 
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We focus on the case of a 2-dimensional grid where the set of groups Q is the set of all horizon- 
tal and vertical half-spaces (see Fig. 1 taken from [1]). As proved in [1, Theorem 3.1], the ^1/^2 
norm Q, sets to zero some groups of variables \\d'^ o y\\^, i.e., some entire horizontal and vertical 
half-spaces of the grid, and therefore induces rectangular nonzero patterns. Note that a broader set 
of convex patterns can be obtained by adding in Q half-planes with other orientations. In practice, 
we use planes with angles which are multiples of j. 




Figure 1 : (Left) The set of blue and green groups with their (not displayed) complements to penal- 
ize to select rectangles. (Right) In red, an example of recovered pattern in this setting. 



Among sparsity inducing regularizations, li is often privileged since it is convex. However, 
so-called concave penalizations, such as penalization by an ia quasi-norm, which are closer to £q 
and penalize more aggressively small coefficients can be prefen^ed, especially in a context where 
the unregularized problem, here dictionary learning is itself non convex. In light of recent work 
showing the advantages of addressing sparse regression problems through concave penalization 
(e.g., see [16]), we therefore generalize to a family of non-convex regularizers as follows: for 
a G (0, 1), we define the quasi-norm for all vectors y € as 

= I E ll^'' ° 1 = II (IM^ ° y\\2)Geg L ' 

where we denote by {\\d^ oy\\^)Q^g g M^^'^' the |^|-tuple composed of the different blocks 
\\d^ o y\\2- We thus replace the (convex) ^1/^2 norm $7 by the (neither convex, nor concave) £a/^2 
quasi-norm fl". Note that this modification impacts the sparsity induced at the level of groups, 
since we have replaced the convex £1 norm by the concave £a quasi-norm. 



3 Optimization 

We consider the optimization of Eq. (1) where we use Q,y = 17" to regularize the dictionary V. We 
discuss in Section 3.3 which norms $7^ we can handle in this optimization framework. 



3.1 Formulation as a sequence of convex problems 

We are now considering Eq. (1) where we take ili, to be 17", that is. 
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X-UV 



+ AE^"(^'') s.t. Vfc, < 1. (2) 



k=l 



Although the minimization problem Eq. (2) is still convex in U for V fixed, the converse is not true 
anymore because of 17". Indeed, the formulation in V is non-differentiable and non-convex. To 
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address this problem, we use the variational equality based on the following lemma that is related^ 
to ideas from [17, 18]: 



Lemma 3.1. Let a G (0, 2) and j3 



2-a 



. For any vector y £ M^, we have the following equality 



\y\\a 



mm 



\Z\\I3, 



and the minimum is uniquely attained for Zj = \yj\'^ " ll^lla ^ i G {1, . . . , p}. 

Proof. Let il^ : z i-^ Sj=i y'j^j'^ + 11-^11/3 continuously differentiable function defined on 

(0, +oo). We have limn^n^^oo VC-z) = +oo and lim^^.^o ^(-z) = +00 if yj / (for yj = 0, note 
that minz>oip{z) = min2>o,Xj=o ^(^))- Thus, the infimum exists and it is attained. Taking the 
derivative w.r.t. Zj (for Zj > 0) leads to the expression of the unique minimum, expression that 
still holds for Zj = 0. □ 

To reformulate problem (2), let us consider the |t/|-tuple {rj'^)Geg S M''^!^' of r-dimensional 
vectors rj^ that satisfy for all /c € {!,..., r} and G € t/, r?^ > 0. It follows from Lemma (3.1) 
that 

If we introduce the matrix ( G W'^'^ defined by"* Cjfe = { Z^ceg G9j(^i )~^}~^' ^^^^ 
obtain 

r ^ 1 

This leads to the following formulation 
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u. V, Q.u{u'')<i 2np 



X-UV 



k=l 



-1 



(3) 



which is equivalent to Eq. (2) and convex with respect to V. 



3.2 Sharing structure among dictionary elements 

So fai^, the regularization quasi-norm 0,°' has been used to induce a structure inside each dictionary 
element taken separately. Nonetheless, some applications may also benefit from a control of the 
structure across dictionary elements. For instance it can be desirable to impose the constraint that r 
dictionary elements share only a few different nonzero patterns. In the context of face recognition, 
this could be relevant to model the variability of faces as the combined variability of several parts, 
with each part having a small support (such as eyes), and having its variance itself explained by 
several dictionary elements (corresponding for example to the color of the eyes). 

To this end, we consider M, a partition of {1, . . . , r}. Imposing that two dictionary elements 
and share the same sparsity pattern is equivalent to imposing that Vf' and V^^ are si- 
multaneously zero or non-zero. Following the approach used for joint feature selection ([19]) 

^^Note that we depart form [17, 18] who consider a quadratic upperbound on the squared norm. We prefer to remain 
in the standard dictionary learning framework where the penalization is not squared. 
"^For the sake of clarity, we do not specify the dependence of on {Ti'^)Gsg- 
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where the ii norm is composed with an £2 norm, we compose the norm ri" with the ^2 norm 
= ||(^^)fceA/||2> of all i^^ entries of each dictionary element of a class M of the partition, 
leading to the regulaiization: 



)i€G,keM 



.Geg 



l/a 



(4) 



In fact, not surprisingly given that similar results hold for the group Lasso [17], it can be shown 
that the above extension is equivalent to the variational formulation 



mm 

u, V, n„((7'=)<i 2np 



1 T 2 

X-UV^ 



,|A1|x|S| 



M£M '-keM 



with class specific variables tjm, C*^' M e M., defined by analogy with r]k and C'^, kG{l, . . . ,r}. 



3.3 Algorithm 

The main optimization procedure described in Algorithm 1 is based on a cyclic optimization over 
the three variables involved, namely (??'')gg5> U and V. We use Lemma (3. 1) to solve Eq. (2) by a 
sequence of problems that are convex in U for fixed V (and conversely, convex in V for fixed U). 
For this sequence of problems, we then present efficient optimization procedures based on block 
coordinate descent (BCD) [20, Section 2.7]. We describe these in detail in Algorithm 1. Note 
that we depart from the approach of [1] who use an active set algorithm. Their approach does not 
indeed allow warm restarts, which is crucial in our alternating optimization scheme. 



Update of {^]^)Geg The update of (ri^)G&g is straightforward (even if the underlying mini- 
mization problem is non-convex), since the minimizer [t]'^)* in Lemma (3.1) is given in closed- 
form. In practice, as in [18], we avoid numerical instabilities near zero with the smoothed update 
^fc iVk)* + with e < 1. 



Update of U The update of U follows the technique suggested by [11]. Each column U'^ of U is 
constrained separately through Qu{U^). Furthermore, if we assume that V and {U^}jjLk are fixed, 
some basic algebra leads to 



arg mm 



X-UV^ 



arg mm 

nu{u'')<i 

arg min 

n„((7fe)<i 



{X-Y^[W]'^V^)V'' 



(5) 
(6) 



which is simply the Euclidian projection IIq^ {w) of w onto the unit ball of O^. Consequently, the 
cost of the BCD update of U depends on how fast we can perform this projection; the ii and I2 
norms are typical cases where the projection can be computed efficiently. In the experiments, we 
take to be the I2 norm. 

In addition, since the function U'^ 
(closed convex) unit ball of 0^, the convergence of the BCD procedure is guaranteed since the 
minimum in Eq. (5) is unique [20, Proposition 2.7.1]. The complete update of U is given in 
Algorithm 1. 



^ ' " 2np II^~^^^IIf continuously differentiable on the 
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Update of V A fairly natural way to update V would be to compute the closed form solutions 
available for each row of V. Indeed, both the loss 2^ — ?7y^||p and the penalization on V are 
separable in the rows of V, leading to p independent ridge-regression problems, implying in turn 
p matrix inversions. 

However, in light of the update of U, we consider again a BCD scheme on the columns of V 
that turns out to be much more efficient, without requiring any non-diagonal matrix inversion. The 
detailed procedure is given in Algorithm 1 . The convergence follows along the same arguments as 
those used for U. 

Algorithm 1 Main optimization procedure for solving Eq. (3). 
Input: Dictionary size r, data matrix X. 
Initialization: Random initialization of U, V. 
while ( stopping criterion not reached ) 

Update {r]'^)c<^g- closed-form solution given by Lemma (3.1). 
Update U by BCD: 

for t = 1 to Tu, tor k = 1 to r: 

^ Iia^{U'' + \\v^\\~\xv^ -UV^V^)). 
Update V by BCD: 

for t = 1 to T^, for A; = 1 to r: 

y '^^Diag {C,^)Bmg(\\U%\c^+np\iylx^U''-VU^U^^U^f^V''). 
Output: Decomposition U, V. 



Our problem is not jointly convex in (?7°)Gg5> U and V , which raises the question of the 
sensitivity of the optimization to its initialization. This point will be discussed in the experiments. 
Section 4. In practice, the stopping criterion relies on the relative decrease (typically 10~^) in the 
cost function in Eq. (2). 

Algorithmic complexity The complexity of Algorithm 1 can be decomposed into 3 terms, cor- 
responding to the update procedures of {i]'^)G'^g^ U and V. We denote by r„ (respectively 
Ty) the number of updates of U (respectively V) in Algorithm 1. First, computing Gee 
and C costs 0{r\g\ + \g\EGeg\G\ + r^^ilG ^ Q\G 3 i|) = 0{pr\G\ ^ p\Q\^). The 
update of U requires 0((p + Tu7i)r'^ + {np + Ci]Tu)r) operations, where Cn is the cost of 
projecting onto the unit ball of Similarly, we get for the update of F a complexity of 
0((n + T„p)r^ + npr). In practice, we notice that the BCD updates for both U and V require only 
few steps, so that we choose = T„ = 3. In our experiments, the algorithmic complexity simpli- 
fies to 0{p^ + max{n,p} + rpmaxjp^/^, n}) times the number of iterations in Algorithm 1. 

Extension to NMF Our formalism does not cover the positivity constraints of non-negative ma- 
trix factorization, but it is straightforward to extend it at the cost of an additional threshold opera- 
tion (to project onto the positive orthant) in the BCD updates of U and V. 

4 Experiments 

We first focus on the application of SSPCA to a face recognition problem and we show that, 
by adding a spai^se structured prior instead of a simple sparse prior, we gain in robustness to 
occlusions. We then apply SSPCA to biological data to study the dynamics of a protein/protein 
complex. 
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Figure 2: Three learned dictionaries of faces with r = 36: NMF (top), SSPCA (middle) and 
shared-SSPCA (bottom) (i.e., SSPCA with \M\ = 12 different patterns of size 3). The dictionary 
elements are sorted in decreasing order of variance explained. While NMF gives sparse spatially 
unconstrained patterns, SSPCA finds convex areas that correspond to more natural face segments. 
SSPCA captures the left/right illuminations in the dataset by recovering symmetric patterns. 

The results we obtain are validated by known properties of the complex. In preliminary exper- 
iments, we considered the exact regularization of [1], i.e., with a = 1, but found that the obtained 
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patterns were not sufficiently sparse and salient. We therefore turned to the setting where the 
parameter a is in (0, 1). In the experiments described in this section we chose a = 0.5. 

4.1 Face recognition 

We first apply SSPCA on the cropped AR Face Database [21] that consists of 2600 face images, 
corresponding to 100 individuals (50 women and 50 men). For each subject, there are 14 non- 
occluded poses and 12 occluded ones (the occlusions are due to sunglasses and scarfs). We reduce 
the resolution of the images from 165x120 to 38x27 for computational reasons. 

Fig. 2 shows examples of learned dictionaries (for r = 36 elements), for NMF, SSPCA 
and SSPCA with shared structure. While NMF finds sparse but spatially unconstrained patterns, 
SSPCA select sparse convex areas that correspond to a more natural segment of faces. For instance, 
meaningful parts such as the mouth and the eyes are recovered by the dictionary. 




10 20 30 40 50 60 70 

Dictionary size 



Figure 3: Correct classification rate vs. dictionary size: each dimensionality reduction technique 
is used with k-NN to classify occluded faces. SSPCA shows better robustness to occlusions. 

We now compare SSPCA, SPCA (as in [12]), PCA and NMF on a face recognition problem. 
We first split the data into 2 parts, the occluded faces and non-occluded ones. For different sizes 
of the dictionary, we apply each of the aforementioned dimensionality reduction techniques to the 
non-occluded faces. Keeping the learned dictionary V , we decompose both non-occluded and 
occluded faces on V. We then classify the occluded faces with a k-nearest-neighbors classifier 
(k-NN), based on the obtained low-dimensional representations. Given the size of the dictionary, 
we choose the number of nearest neighbor(s) and the amount of regularization A by 5 -fold cross- 
validation^ . 

The formulations of NMF, SPCA and SSPCA are non-convex and as a consequence, the local 
minima reached by those methods are sensitive to the initialization. Thus, after having selected the 

""In the 5-fold cross-validation, the number of nearest neighbor(s) is searched in {1,3,5} while log2(A) is in 
{4, 6, 8, ... , 18}. For the dictionary, we consider the sizes r € {10, 20, 30, 40, 50, 60, 70}. 
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parameters by cross-validation, we run each algorithm 20 times with different initializations on the 
non-occluded faces, divided into a training (900 instances) and validation set (500 instances) and 
take the model with the best classification score. We summarize the results in Fig. 3. We denote 
by shared-SSPCA (resp. shared-SPCA) the models where we impose, on top of the structure of 
0,", to have only 10 different nonzero patterns among the learned dictionaries (see Section 3.2). 

As a baseline, we also plot the classification score that we obtain when we directly apply k- 
NN on the raw data, without preprocessing. Because of its local dictionary, SSPCA proves to 
be more robust to occlusions and therefore outperforms the other methods on this classification 
task. On the other hand, SPCA, that yields sparsity without a structure prior, performs poorly. 
Sharing structure across the dictionary elements (see Section 3.2) seems to help SPCA for which 
no structure information is otherwise available. 

The goal of our paper is not to compete with state-of-the-art techniques of face recognition, 
but to demonstrate the improvement obtained between ii and more structured norms. We could 
still improve upon our results using non-linear classification (e.g., with a SVM) or by refining our 
features (e.g., with a Laplacian filter). 

4.2 Protein complex dynamics 

Understanding the dynamics of protein complexes is important since conformational changes of 
the complex are responsible for modification of the biological function of the proteins in the com- 
plex. For the EF-CAM complex we consider, it is of particular interest to study the interaction 
between EF (adenyl cyclase of Bacillus anthracis) and CAM (calmodulin) to find new ways to 
block the action of anthrax [22]. 

In our experiment, we consider 12000 successive positions of the 619 residues of the EF-CAM 
complex, each residue being represented by its position in (i.e., a total of 1857 variables). 
We look for dictionary elements that explain the dynamics of the complex, with the constraint 
that these dictionary elements have to be small convex regions in space. Indeed, the complex is 
comprised of several functional domains (see Fig. 4) whose spatial structure has to be preserved 
by our decomposition. 

We use the norm il" (and Q) to take into account the spatial configuration. Thus, we naturally 
extend the groups designed for a 2-dimensional grid (see Fig. 1) to a 3-dimensional setting. Since 
one residue corresponds to 3 variables, we could either (1) aggregate these variables into a single 
one and consider a 619-dimensional problem, or (2) we could use Section 3.2 to force, for each 
residue, the decompositions of all three coordinates to share the same support, i.e., in a 1857- 
dimensional problem. This second method has given us more satisfactory results. 

We only present results on a small dictionary (see Fig. 4 with r = 3). As a heuristic to pick A, 
we maximize | Ufc=iSupp(V'') P/(pI]fc=i|supp(V'°) |) to select dictionary elements that cover pretty 
well the complex, without too many overlapping ai^eas. 

We retrieve groups of residues that match known energetically stable substructures of the com- 
plex [22] . In particular, we recover the two tails of the complex and the interface between EF and 
CAM where the two proteins bind. Finally, we also run our method on the same EF-CAM complex 
perturbed by (2 and 4) calcium elements. Interestingly, we observe stable decompositions, which 
is in agreement with the analysis of [22]. 
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Figure 4: (Top left) Entire protein with biological domains highlighted in different colors. The 
two blue parts represent the CAM protein, while the rest of the complex corresponds to EF. (Top 
right, bottom left/right) Dictionary of size r = 3 found by SSPCA with the same color code. 

5 Conclusions 

We proposed to apply a non-convex variant of the regularization introduced by [1] to the prob- 
lem of structured sparse dictionary learning. We present an efficient block-coordinate descent 
algorithm with closed-form updates. For face recognition, the dictionaries learned have increased 
robustness to occlusions compared to NMF An application to the analysis of protein complexes 
reveals biologically meaningful structures of the complex. As future directions, we plan to re- 
fine our optimization scheme to better exploit sparsity. We also intend to apply this sti'uctured 
spai^sity-inducing norm for multi-task learning, in order to take advantage of the structure between 
tasks. 
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